1. Technical Field
The present invention relates to a spectral image processor including a camera, a diffraction grating and a processor to detect and classify light sources in real time. Embodiments of the present invention are applicable to driver assistance systems (DAS). Specifically, oncoming vehicle headlights, leading vehicle taillights and streetlights, pedestrians, industrial lights and road signs are detected using a spectrum of the light sources. The spectral image processor allows for multiple DAS applications to be run simultaneously.
2. Description of Related Art
It is desirable for a vehicle control system to have the ability to classify various light sources including headlamps, taillights, street-lamps, house lights, and industrial lights. Schofield et al. (U.S. Pat. No. 6,831,261), disclose a headlight control device capable of identifying spectral characteristics of light sources using a camera equipped with one or more spectral filters. The spectral filters which provide the ability to distinguish red and white lights for example in the case of taillights and headlamps, also limit the use of the simultaneous use of the device of the same image frames for other driver assistance applications, such as lane departure warning and road sign detection.
Although white light of a halogen headlamp and a streetlight appear to both be white, their spectral composition is quite different. The low pressure sodium and high pressure sodium arc lamps used for street-lamps and industrial lights has a very distinct spectral pattern than that of a headlamp. Other white lights and fluorescent lights have different spectral characteristics. Since these ‘white’ light sources appear the same in a color camera with for instance red/green/blue color filters. (RGB) camera the different white light sources cannot be always be easily differentiated based on discrete color information.
Reference is now made to FIG. 1 and FIG. 2, which illustrate a vehicle control system 16 including a camera and image sensor 12 mounted in a vehicle 18 imaging a field of view in the forward or rear direction. Image sensor 12 typically delivers images in real time and the images are captured in a time series of image frames 15. An image processor 14 is used to process image frames 15 to perform one of a number of prior art vehicle controls. Prior art vehicle control systems include forward collision warning systems, lane departure warning systems, road sign detection, pedestrian detection and headlight control systems.
Headlight control is described in a pending US application US2007/0221822 of the present inventor. US application 2007/0221822 discloses image sensor 12 which captures image frames 15 consecutively in real time for headlight detection in conjunction with other driver control systems. The light source is typically one or more of headlights from an oncoming vehicle, taillights of a leading vehicle, street signs and/or traffic signs. The image spots are characterized in terms of position, intensity, size, and shape, but not by color, i.e. color filters are not used in the camera. Since color filters are not used in the camera, the image frames are available for sharing between multiple driver assistance system without sacrificing performance.
Collision Warning is disclosed in U.S. Pat. No. 7,113,867 by the present inventor. Time to collision is determined based on information from multiple images 15 captured in real time using camera 12 mounted in vehicle 18.
Lane Departure Warning (LDW) is disclosed in U.S. Pat. No. 7,151,996 by the present inventor. If a moving vehicle has inadvertently moved out of its lane of travel based on image information from images 15 from forward looking camera 12, then the system signals the driver accordingly.
Ego-motion estimation is disclosed in U.S. Pat. No. 6,704,621 by the present inventor. Image information is received from images 15 recorded as the vehicle moves along a roadway. The image information is processed to generate an ego-motion estimate of the vehicle, including the translation of the vehicle in the forward direction and the rotation. Vehicle control systems, such as disclosed in U.S. Pat. No. 6,831,261 which rely on changing exposure parameters (ie, aperture, exposure, magnification, etc) in order to detect headlights have a difficult time maintaining other control systems which rely on the same camera, e.g. Lane Departure Warning, Forward Collision Warning, etc. As a result of changing exposure parameters half or more of the (possibly critical) frames may not be available for the other control systems. This greatly affects performance of the other control systems.
Hence, since in the vehicle headlight control system as disclosed in U.S. Pat. No. 6,831,261 (or in any other disclosure where special control is required of camera settings including, aperture, exposure time and magnification), the same camera cannot be conveniently used for other simultaneously operable vehicle control systems such as lane departure warning or collision warning.
Additionally, the use of color cameras with infrared filters required to achieve good spectral separation reduces imaging sensitivity by a factor of six or more. A reduction in sensitivity by such a factor has an adverse impact on other vehicle control application such as LDW performance in dark scenes. The presence of an infrared filter also negates the use of the camera as a near infrared sensor for applications, such as pedestrian detection. Thus, headlight control systems which make strong use of color or spectral analysis with the use of color or absorptive or dichroic filters in the captured images (such as in U.S. Pat. No. 6,831,261) will tend not be compatible with other applications without sacrificing performance.
Thus there is a need for, and it would be advantageous to have, a spectral processor adapted to identify various light sources by their spectral composition without adversely affecting use of the same image frames by other driver assistance applications.
Support vector machines (SVMs) are a set of related supervised learning methods used for classification and regression. Support vector machines (SVMs) belong to a family of generalized linear classifiers Support vector machines (SVMs) can also be considered a special case of Tikhonov regularization. A special property of SVMs is that they simultaneously minimize the empirical classification error and maximize the geometric margin; hence they are also known as maximum margin classifiers.
Viewing the input data as two sets of vectors in an n-dimensional space, an SVM will construct a separating hyper-plane in that space, one which maximizes the “margin” between the two data sets. To calculate the margin, two parallel hyper-planes are constructed, one on each side of the separating one, which are “pushed up against” the two data sets. Intuitively, a good separation is achieved by the hyper-plane that has the largest distance to the neighboring data points of both classes. The hope is that, the larger the margin or distance between these parallel hyper-planes, the better the generalization error of the classifier will be.